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Abstract 


The objective of the research developed and presented in this document was to statistically assess 
turbulence hazard detection performance employing airborne pulse Doppler radar systems. The FAA 
certification methodology for forward-looking airborne turbulence radars will require estimating the 
probabilities of missed and false hazard indications under operational conditions. Analytical approaches 
must be used due to the near impossibility of obtaining sufficient statistics experimentally. This report 
describes an end-to-end analytical technique for estimating these probabilities for Enhanced Turbulence 
(E-Turb) Radar systems under noise-limited conditions, for a variety of aircraft types, as defined in FAA 
Technical Standard Order Airborne Weather Radio Detection and Ranging (RADAR) Equipment. This 
technique provides for one means, but not the only means, by which an applicant can demonstrate 
compliance to the FAA directed ATDS Working Group performance requirements. Turbulence hazard 
algorithms were developed that derived predictive estimates of aircraft hazards from basic radar 
observables. These algorithms were designed to prevent false turbulence indications while accurately 
predicting areas of elevated turbulence risks to aircraft, passengers, and crew; and were successfully flight 
tested on a NASA B757-200 and a Delta Air Lines’ B737-800. Application of this defined methodology 
for calculating the probability of missed and false hazard indications taking into account the effect of the 
various algorithms used, is demonstrated for representative transport aircraft and radar performance 
characteristics. 


Subject Terms 

Aviation Safety, Turbulence, Turbulence Detection, Turbulence Encounters, Turbulence Reporting, 
Doppler Radar, Enhanced Airborne Weather Radar, Enhanced Turbulence (E-Turb), Airborne Remote 
Sensing, Detection Probability, Turbulence Flazard Metric, Weather, Safety. 
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1. Introduction and Background 

Turbulence has been identified as a significant operational aviation safety hazard during all phases of 
flight. Since 1998, the National Aeronautics and Space Administration (NASA) has conducted and 
funded research in the areas of turbulence detection and avoidance. Subsequently, NASA contracted 
AeroTech Research to conduct research and develop turbulence detection and avoidance systems in 
support of NASA’s Aviation Safety and Security Program’s (AvSSP) overall goal to “develop and 
demonstrate technologies that contribute to a reduction in aviation accident and fatality rates.” From 1998 
to 2003, AeroTech Research (and other NASA partners) developed the concepts and initial algorithms of 
various safety-related technologies under the NASA Turbulence Prediction and Warning Systems 
(TPAWS) element of the Weather Accident Prevention (WxAP) Project within AvSSP. The WxAP 
Program’s three objectives to support the goal of the AvSSP were: 

1. Develop technologies and methods that will provide pilots with sufficiently accurate, timely, and 
intuitive information during the en-route phase of flight, which, if implemented, will enable a 25- 
50% reduction in aircraft accidents attributable to lack of weather situational awareness. 

2. Develop communications technologies that will provide a 3- to 5-fold increase in datalink system 
capacity, throughput, and connectivity for disseminating strategic weather information between 
the flight deck and the ground, which, if implemented along with other supporting technologies, 
will enable a 25-50% reduction in aircraft accidents attributable to lack of weather situational 
awareness. 

3. Develop turbulence prediction technologies, hazard metric methods, and mitigation procedures to 
enable a 25-50% reduction in turbulence-related injuries. 

Under the TPAWS element, the Enhanced Turbulence (E-Turb) Airborne Radar came to the forefront as a 
technology that was realizable and a significant contributor to meeting the TPAWS goal to “provide 
airborne centric technology for detection and cockpit display of hazardous turbulence” that when 
developed would “enable about a 50% reduction in injuries attributable to the lack of turbulence 
situational awareness.” This technology was further developed and evaluated both in simulations and 
flight experiments onboard NASA’s B757-200 ARIES Research Aircraft. The NASA flight experiments 
proved that the E-Turb Radar technology performed as designed and would provide the basis for much 
improved turbulence detection and awareness. 

Engineering issues with the NASA B757 aircraft in late 2003 caused the cancellation of the TPAWS 
flight experiments for the E-Turb Radar technology. Realizing the importance of the research and needing 
a way to properly evaluate the technology, NASA and AeroTech Research sought collaboration within 
the aviation industry. 

In August of 2003, a two-month feasibility study was initiated by NASA to develop the content and 
structure of a potential In-Service Evaluation (ISE) of the E-Turb Radar technology. The results of that 
study established a two-year ISE of the technology with the participation of Delta Air Lines (DAL) and 
Rockwell Collins. Reference [1] provides an overview and summary of the efforts, analyses, and results 
of the ISE initiatives. The specific objectives of the E-Turb Radar ISE were to implement and integrate E- 
Turb Radar algorithms within an airborne radar, install the radar onboard a commercial transport aircraft, 
and to evaluate the performance and effectiveness of the enhanced radar system in actual transport 
aviation operational environments over a reasonable timeframe. During the two year ISE, the E-Turb 
Radar algorithms were implemented into a WXR-2100 Multiscan™ radar, which was installed into a 
B737-800 aircraft and flown over 3000 flight hours in operational service. The E-Turb Radar 1) 
performed as per design and intended function - provided improved, objective turbulence hazard 
detection and awareness relevant to the specific aircraft and its flight conditions; 2) received positive 
feedback from the flight crews; and 3) based on collected data and convincing evidence, was used by 
crews to avoid turbulence. The collected data also indicated strong correlation between E-Turb Radar 
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predicted loads and experienced loads when avoidance was not possible. Currently (June 2009), the ISE 
E-Turb Radar is still flying in operational service on the Delta B737-800. 


The success of the E-Turb Radar ISE led NASA to direct AeroTech Research to investigate additional 
enhancements to the E-Turb Radar, an E-Turb capability with small radar antennas, a “one-size-fits-all” 
option for retrofitting E-Turb onto already installed radars, and an E-Turb Radar capability for regional 
jets and business jets (low wing loading aircraft). The results of this follow-on research indicated the 
feasibility of the application of E-Turb technology to a broad range of aircraft and radars. Figure 1 
illustrates a timeline of the NASA-sponsored E-Turb Radar development activities. 


ANALYSIS & SIMULATION 

NASA B7S7 FLT. EXP.’s 

IN-SERVICE EVALUATION 

ENHANCEMENT 



1998 


2000 


2004 


2006 


2008 


Figure 1: Timeline of NASA Sponsored E-Turb Radar Research and Development 

Several avionics manufacturers are currently developing forward-looking turbulence detection systems 
for commercial aircraft using new airborne pulse Doppler radar technology. The difficulty in evaluating 
these systems under realistic hazardous turbulence and operational conditions has raised new certification 
issues. A significant part of the certification process will require estimating system performance using a 
combination of analytical studies, computer simulations, and flight tests. 

In order to facilitate the certification process, the Federal Aviation Administration (FAA) formed and is 
leading an Airborne Turbulence Detection System (ATDS) Working Group inclusive of NASA, 
AeroTech Research, Rockwell Collins, Honeywell, Boeing, and Airbus personnel to determine the 
Minimum Operational Performance Standards (MOPS) and establish the certification methodology. In the 
proposed certification requirements as documented by the FAA in the ATDS Working Group’s MOPS, 
minimum probabilities of missed and false turbulence indications have been specified. Each manufacturer 
seeking FAA certification approval for a specific aircraft application must make estimates of these 
probabilities for one of the three aircraft classes as defined in Reference [2]. The ATDS Working Group 
and the FAA established a three aircraft class differentiation based on wing loading criteria (aircraft 
weight divided by wing reference area). The specific classes are defined as: 

• Class A -80 to 135 lbs/ft 2 

• Class B- 60 to 100 lbs/ft 2 

• Class C - 30 to 70 lbs/ft 2 
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The methodology and examples presented in this document focus on the Class A (large transport category 
aircraft), but results are applicable to the other two classes of aircraft. In the following sections, 
algorithms used for estimating turbulence hazards from basic radar measurements are described and a 
methodology for estimating probabilities of missed and false turbulence indications under noise limited 
conditions is presented. This end-to-end methodology includes the formulation, development, and 
verification of the necessary governing equations and computational tools to calculate probabilistic 
detection performance for g-based turbulence prediction systems employing airborne radar observables. 
The formulation considers aircraft characteristics and associated errors as well as airborne radar 
measurement characteristics and detection errors. Overall end-to-end statistical performance of the g- 
based turbulence system depends, in general, on the product of two random variables and the individual 
statistical characteristics of both random variables. Although the methodology described herein is specific 
to the detection techniques and algorithms used, it serves as a guide for calculations on other systems 
proposed for certification by various manufacturers of radar based turbulence detection systems. 

2. Aircraft Centric Turbulence Hazard Metric Definition 

Early in the NASA TPAWS Program, there was an effort to identify a metric which quantifies a 
turbulence hazard to a commercial transport aircraft. The primary requirements were: 

1. It should unambiguously represent the intensity of the turbulence hazard based on accelerations, 
which result in injuries and damage to an aircraft. 

2. It should not depend on the atmospheric phenomenon that produces the effect on the aircraft. 

3. It could be related to measurements or observables made by various forward-looking airborne 
sensors (e.g., radar, lidar, etc.). 

4. It could be measured by in situ sensors onboard an aircraft; thereby, providing a “truth” 
measurement to assess the performance of the forward-looking sensors. 

5. It could be readily scaled from one aircraft to another based on accepted physics. (Reference [3]) 

The metric that best satisfies these conditions was a running 5-second windowed root mean square (RMS) 
of the aircraft vertical acceleration, denoted by a An . The metric was refined in simulations and several 
sets of flight experiments on NASA’s B757-200 research aircraft under TPAWS. There is plausible 
justification for this choice of metric given the longitudinal response characteristics and operating speeds 
of transport category aircraft. The selection of five seconds was based on two key considerations: 

• The need to balance between 1) a sample window small enough to adequately resolve small scale 
turbulence that affect aircraft through induced g-loads and 2) an accelerometer measurement 
sample size large enough to calculate an RMS with acceptably small random error; and 

• Five seconds corresponds to the one-kilometer spatial average used for the airborne radar 
turbulence signal processing, based on typical cruise airspeeds. Therefore, there was consistency 
between the forward-looking airborne sensor and the in situ accelerometer measurements. 

The a An parameter can also be related to the peak accelerations experienced by an aircraft. Figure 2 
shows turbulence encounters from historical flight data; including data collected during previous NASA 
flight tests, the E-Turb Radar In-Service Evaluation, National Transportation and Safety Board (NTSB) 
accident investigations, and several other airline incidents and accidents. A linear regression applied to 
this data yields a correlation of 95%. This data clearly demonstrates that the <J An parameter can be used as 
a surrogate for peak loads. 
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Based on Measurements for606Turbulence EncounterCases 

Data Sources 

□ 18 ICAO Events 

• 49 NASA Events (FY02) 

O 1 8 NASA Events (FYOO) 

A 10 NTSB Accidents 

• 6 FOQA Incidents 
A UAL 747 Pacific Accident 
+ DAL 3708 ISE (495 Events) 
■ Recent Severe Encounters 

Regression 

* Cases Chosen for 
Detail Modeling to 
Support FAA Radar 
Certification Effort 


0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 

Peak a An (g's) 

Figure 2: Correlation between Peak Load and Peak cr A „ (5 sec. window) 

Analyses were conducted to select thresholds of a An that could be used to define the various levels of 
turbulence intensity. The selection of thresholds was hampered by the lack of clear, objective data relating 
the a An parameter to the usual subjective descriptions of light, moderate, and severe turbulence. The FAA 
Aeronautical Information Manual (AIM) states that during severe turbulence “occupants are forced 
violently against seat belts ...” and/or “the aircraft may be momentarily out of control.” Based on this, a 
threshold of a An = 0.3g was conservatively chosen as the lower limit of severe turbulence. These data are 
consistent with thresholds defined in the Forecasting Guide on Turbulence Intensity (Reference [4]). The 
FAA in Reference [2] has specified the g-based metric for representing enhanced turbulence predictions 
based on radar observables. 

It is important to note that no scientific basis exists to be exact in the categorizations of the turbulence 
intensities. The exact subjective assessment of turbulence is controversial and has not been definitively 
determined. The turbulence metric thresholds selected can be used as a basis for warning pilots and 
dispatchers of potential safety hazards. The following categorization was adopted for the E-Turb Radar 
analysis within this report. 

Oa n ^ O.lg 
0.1 < cr An < 0.2g 
0.2 < a An < 0.3g 
0.3 < a An < 0.6g 
0.6 < a An 

3. Hazard Prediction Algorithm Design 

As discussed in Section 2, the overall turbulence hazard to an aircraft is characterized by a numerical 
value (g-units) of RMS vertical acceleration excursions relative to lg quiescent flight. This metric 
predicts impending impact on an aircraft and its occupants as well as unacceptable flight path excursions, 
and is based on accepted fundamentals of flight mechanics. Using these fundamental concepts, the 
quantitative impact of turbulence on an aircraft can be mathematically defined. 


Light Turbulence 
Moderate Turbulence 
Moderate to Severe Turbulence 
Severe Turbulence 
Extreme Turbulence 
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Aircraft respond to lift changes induced by deviations of angle of attack, which result from atmospheric 
velocity disturbances. Such lift changes produce aircraft loads, which are proportional to point variance of 
the turbulence velocity field. In order to predict aircraft loads based on radar observables, a relationship 
between the radar spectrum width measurement, point variance of turbulence intensity, and a A n is needed. 
Such a relationship, based on the physics of the radar measurement process, allows estimation of 
turbulence point variance from radar observables. The structure of a generic hazard prediction algorithm 
based on airborne radar observables can be approximated by: 

- _ I PAn 1 _ [M 2 (%)] 0S 

An tu-nit a J taM e J (rT g (r)> (1) 


where: 


d An = Predicted value of RMS vertical load excursion from lg (g’s) 

ffAn — = Aircraft scale (conversion) factor (g’s/m/s) 

Yunita w \ tame 

M 2 (x) = Radar 2 nd moment / spectrum width product (nr/s 2 ) 

V ( a v ( r )) I a = Radar pulse volume compensation factor based on theory (non-dimensional) 

In the above equation, er w is defined as the standard deviation of the vertical component of the turbulent 
wind field and a is the RMS intensity of the turbulent wind field based on one-dimensional von Karman 
energy spectra. A specific point in radar range-azimuth space is defined by x. It is assumed that a 
provides a close approximation for the standard deviation of the component of the turbulent wind field in 
the horizontal direction (er u ) as viewed along a radar radial. For evaluation of system performance, an 
acceptable assumption is that er w ~ This local isotropy assumption, when considering the physical 
dimensions of the radar resolution volume, is further justified by analysis of NASA flight test data found 
in Reference [5]. 

The aircraft-scaling factor in general depends on aircraft altitude, airspeed, and weight. When considering 
conversion of radar measurement products to aircraft load prediction, two distinct cases may be 
implemented. The first case directly applies existing turbulence hazard table data developed under NASA 
sponsorship (see Section 6). Data exists for a variety of aircraft types ranging from A320-200 to B747- 
400 for altitudes of 5,000 feet to service ceilings, over the entire operational weight range for each 
aircraft. This system implementation requires aircraft weight to be interfaced to the radar and is referred 
to as the “weight strapped” configuration. The design philosophy for the second case seeks to remove 
weight (wing loading) dependence while achieving an acceptable error budget, thus reducing overall 
system cost and implementation complexity. For this case, a statistical algorithm design is required for the 
g-based turbulence system consistent with the design criteria that aircraft weight shall not be “strapped” 
to the radar system. This notion has been referred to as the “one-size-fits-all” system configuration. 
Further details of the “one-size-fits-all” approach including relevant data for application is discussed in 
Section 6 and 7 of this report. 

If an applicant seeking certification approval elects to include radar pulse volume compensation for their 
system implementation, then such considerations should be included when demonstrating compliance to 
performance required in sections 1.3(a) and 1.3(b) of the ATDS Working Group’s MOPS. The 
performance criteria documented in this MOPS was based on a consensus formed over several years by 
the ATDS Working Group. The technical necessity for this compensation factor stems from the fact that 
spatial spectra of turbulence velocities are filtered by the radar resolution volume characteristics (for 
specific radar & antenna design parameters) and thus produces attenuation in observed turbulence 
intensity estimates derived from radar measurements. This phenomena is referred to as pulse volume 
filtering, the physics of which describe how turbulent kinetic energy is partitioned between subresolution 
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volume scales and scales larger than the radar resolution volume. In effect, a radar system acts as a high 
pass filter measurement device. The key determinant is the pass band of the filter process, which is a 
function of the specific radar performance characteristics and configuration considered. The filtering 
impact is more pronounced for larger antennas than smaller antenna systems. The governing equations for 
calculating the compensation factor including representative numerical results for selected radar 
parameters are presented in Appendix I. The compensation factor is range dependent and is 
approximately equal to unity at longer ranges and decreases as range decreases. 

For purposes of simplicity and clarity of notation in the subsequent text, the following definitions are 
introduced: 


z = RMS predicted vertical load from lg reference (g’s) 

x = Aircraft scaling (conversion) factor (g’s/m/s) 

y = Radar 2 nd moment/spectrum width product (m/s) 

k = 1/ pulse volume compensation factor (non-dimensional) 

Considering the revised nomenclature, Figure 3 shows a diagram of the E-Turb Radar system concept. 
Real-time values of aircraft altitude, airspeed, and weight (if “strapped”) are used in the hazard prediction 
algorithm to scale the 2 nd moment/ spectrum width to a predicted a A „. The 2 nd moment measurement and 
predicted load products are available as a function of range and azimuth over the scanning domain of the 
radar antenna and (i,y) designates a specific range-azimuth intersection. 



Raw Doppler Turbulence Moments Aircraft Hazard 



Figure 3: E-Turb Radar System Concept 

4. Probability Calculations 

The key objective of the material presented in this section is to formulate, develop, and provide initial 
verification of the necessary governing equations and computational tools to establish probabilistic 
detection performance for g-based turbulence prediction systems employing airborne radar observables. 
The formulation considers aircraft characteristics and associated errors (x-component) as well as airborne 
radar measurement characteristics and detection errors (y-component). The aircraft turbulence induced 
load conversion factor, x, has units of g’s/m/s and the radar spectrum width estimate, y, has units of m/s. 
Overall end-to-end statistical performance (z = kxy in units of g) of the g-based turbulence system 
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depends on the product of two random variables and the individual statistical characteristics of both the x 
and y random variables. The developed methodology, which is consistent with the requirements of the 
ATDS Working Group’s MOPS, as well as the governing equations for system performance hypothesis 
testing including initial results, was presented at the FAA-ATDS workshops held in March and August of 
2007. The material presented at the workshops included a method for determining the turbulence system 
cockpit display threshold, consistent with realistic nuisance and must detect/indicate probabilities. 

4.1 Formulation of Governing Equations 

Consider the random variable z = xy where x and y are positive random variables and assumed 
independent with a joint density function /(x,y) = / % (x)/ y (y). The x and y random variables and their 
associated probability density functions (PDF) are restricted by physics to x > 0 andy > 0. It is assumed 
that the radar spectrum width has been pulse volume compensated. Explicit modification of the equations 
found herein for inclusion of pulse volume compensation is provided in Appendix II. The objective is to 
calculate the PDF, the cumulative distribution function (CDF), and the first three moments of the random 
variable z given the process z = xy. Based on accepted fundamentals of probability theory (Reference 
[6]), the CDF and PDF of z for any random outcome ( are defined respectively as: 


F z (z) = Pr(C < z) = ff D(z) f(x, y)dxdy 

(2) 

p z (z|z > 0)dz = JJ AD(z) / (x, y)dxdy 

(3) 


Where the region D (z) is the region in the x — y plane bounded by the hyperbola z = xy for fixed 
positive values of z and the +x and +y axes. This region lies in the first quadrant of the x — y plane and 
is characterized by the condition x < z/y for fixed non-zero positive values of z. The region AD(z') is 
defined such that z < xy < z + dz is the region bounded by the hyperbola’s x < z/y and x < (z + 
dz)/ y. The coordinates of a point in this region are (z/y, y) and the area of a differential equals dydz/y. 
For the conditions and definitions stated above we obtain: 


F z (z) = Pr(/ < z) = J 0 °°/ y (y) J 0 Z/y f x {x)dxdy 


(4) 


••• F z( z ) = So fy(y) G(z/y)dy 


(5) 


The probability density function of z can be found by appropriate integration over the region AD(z) or by 
differentiating the cumulative distribution function of Equation (4) with respect to z. Therefore: 


f ■ . M dF z (z) r °° r f \ dG(z/y) , r co fy(y)f x (z/y) 

p z (z|z>0) =^— = Nj 0 fy{y) — dy = N j Q 3-— dy 


( 6 ) 


r oo 

Where N is defined as a normalization factor such that J Q dF z (z) = 1. Analysis of Equations (5) and (6) 
shows that: 


F z (z) increases monotonically as a function of z for 0 < z < oo 


(7) 


N = 


l 

i-G(o) 


( 8 ) 


F z i oo) = 1 


(9) 
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( 10 ) 


J 0 °°p z (z|z > 0 )dz = 1 

p z (z|z > 0) > 0 (11) 

Pr(z x < ( < z 2 ) = Q p z (z | z > 0 )dz (12) 

Probability theory requires all the above properties to be true in order that the z cumulative distribution 
function and related probability density function produce legitimate statistics for the z = xy random 
process. It should be noted that all the above integrals should be interpreted as Stieltjes integrals 
(Reference [6]), which admits a finite or denumerable number of discontinuities and impulses in the 
integrands. This is required for all probability to lie on the positive z-axis, since radar pulse pair 
estimators of spectrum width can return negative or zero values and F z (0) may not be zero. 

Conventionally defined statistics for mean and variance can be calculated once the z-PDF as given by 
Equation (6) is known: 


dz = J 0 °° z Pz( z l z ^ 0)dz = Nfi x fi y (13) 

°z = Jo°° z 2 p z (z|z > 0 )dz - p| = N 2 [(o- x cr y ) 2 + (p*er y ) 2 + (p y cr x ) 2 ] (14) 

The p's represent the means for the x and y random variables and the a's are the standard deviations of x 
and y respectively. 

Three different approaches have been used to derive the above results with complete agreement between 
the methods. The authors believe the formulation outlined above offers computational advantages. In 
order to verify the above equations, several Monte-Carlo simulations of the above process have been 
conducted, with excellent agreement between theoretical and simulation results. Some results and 
comparisons between theory and simulation are shown in the following sections. 

4.2 Verification of Equation (5) and Equation (6) 

4.2.1 Conditions Defined for Monte-Carlo Simulation 

Aircraft flight conditions (i.e. h = 10,000 ft, airspeed = 250-290 kias) were selected for Class A aircraft 
systems and 20,000 operational sorties were simulated. The “one-size-fits-all” algorithm concept was 
assumed and statistical uncertainties in x are due to the fact that for each simulated operation the 
particular x -value was randomly selected from a distribution derived from aircraft operational weight 
data. The “one-size-fits-all” design value for x is based on a reference wing loading and varies as a 
function of altitude and speed. Specific x -parameters are: (see Figure 8) 


p x = 0.04662 g/m/s 

(15) 

o x = 0.004256 g/m/s 

(16) 

Note: o x error/[i x ^ 0.1 

(17) 


Given that a Gaussian distribution closely approximates the x-distribution, for purposes of simulation we 
assume: 


x random variable is N(p. x , o x ) 


(18) 



fxM = 


i 

V 27 T(T X 



(x-n x ) 2 \ 
2(a x y ) 


(19) 


4.2.2 Radar y-Component of Hazard Algorithm 

For purposes of simulation and insufficient specific information from the ATDS industry team, the 
following assumptions are made. Specific y parameters for the radar 2 nd moment observable (spectrum 
width) are: 


g. y = 5 m/s 

(20) 

o y = 1 m/s 

(21) 

Note: Oy/[iy ^ 0.2 

(22) 

The parameters chosen above are indicative of a relatively high radar signal-to-noise ratio (SNR). The 
specific form of the radar y probability density function will play a critical role in establishing a suitable 
system threshold for z and in assigning probabilities to nuisance and must detect/indicate requirements. 
For simulation purposes the following assumptions are made: 

y random variable is N(fi y , a y ) 

(23) 

fyW = V2i„, ex p{ w) 

(24) 


4.2.3 Monte-Carlo Simulation Results 

Given the terms of reference as stipulated above (both theory and simulation), a comparison of theoretical 
(calculated from Equations (5) and (6)) and Monte-Carlo simulation results are shown in Figure 4 and 
Figure 5. 
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Figure 4: PDF Comparison for 20,000 Flight Operations 


Table 1: Comparison of z Statistics 


Measure 

Simulation 

Theory 

Zeroth Moment 

1.0 

1.0 

Mean 

0.23329 g’s 

0.23131 g’s 

Standard Deviation 

0.05139 g’s 

0.05142 g’s 


Comparison of the cumulative probability distribution functions is shown in Figure 5. 
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Figure 5: CDF Comparison for 20,000 Flight Operations 

The comparisons shown between Monte-Carlo simulation and calculated results based on Equations (5) 
and (6) provide convincing evidence as to the validity of the governing equations as developed above. 


4.2.4 Aircraft Weight “Strapped” Condition 

Reduction of the z-PDF (Equation (6)) for the case where aircraft weight is “strapped” to the radar is 
developed below. For this case, the x-PDF is characterized by the Dirac delta function such that the 
following are true: 


/*(*) = S(x - x°); f x (x)dx = 1 (25) 

y-x = x °; ° x = 0 (26) 

The variable x° represents a particular aircraft hazard table point-set value which in general depends on 
aircraft type, speed, altitude, and weight. These tables were developed under NASA contract, and 
documentation describing their development and use is presented in Section 6. For the case considered, 
Equation (6) takes the following form: 


Pz(z|z > 0) = 


r r oo / y (y)«g-*°) 

i 


(27) 


For evaluation of the integral as defined by Equation (27), it is convenient to use the following property 
associated with Dirac delta functions 


fah(i)8(g(i))dt 


Ht ° ) 
D'(t°)l 


(28) 
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provided g(t) = 0 has a single root t° in the interval a < t < b. Examining Equation (27) we conclude 
g(y) = z/y — x° with a single root at y° = z/x° for fixed positive values for z and x° . The derivative 
of g(y) evaluated aty° is (x°) 2 /z; therefore evaluating Equation (27) yields: 


Pz(z|z > 0) = 


l 

X°(l-F y (0)) 



(29) 


Although the above result is well known in the case where x is deterministic, it is instructive to derive the 
result from the more general equation involving the product of two random variables, which is the case 
relevant to the turbulence radar for the “one-size-fits-all” algorithm design. If the y-PDF is Gaussian as 
assumed in the above numerical calculations, Equation (29) reduces to: 


Pz(z|z > 0) = 


N 

^2Ex°a y eXP 


( 


(Z — %V y ) 2 \ 

2(X 0 <Jy) 2 ) 


(30) 


where the mean and standard deviation of z is: 


g z = Nx°g y 

(31) 

b* 

o 

X 

S 

II 

b N 

(32) 


Results have been calculated comparing the “one-size-fits-all” algorithm statistical design philosophy to a 
weight “strapped” implementation. Results of this analysis have been presented to the FAA-ATDS 
working group. The overall end-to-end methodology for calculating the must detect/indicate and must not 
indicate/nuisance probabilities as required by the ATDS Working Group’s MOPS, with application to 
specific Class A aircraft systems and flight conditions, is shown in Section 9 of this report. 

5. Calculation of Nuisance and Missed Detection Probabilities 

Discussion of a probabilistic theoretic technique for calculation of must indicate and must not indicate 
probability is provided in this section. The technique discussed is based on accepted hypothesis testing 
methods and utilizes the basic equations developed in Section 4. E-Turb Radar pass/fail statistical 
performance criteria for Class A aircraft systems, as specified in sections 1.3(a) and 1.3(b) of the ATDS 
Working Group MOPS, requires consideration of two cases. Section 1.3(a) states that the radar shall 
indicate turbulence that corresponds to a <r An of 0.3g for weather targets with reflectivity greater than or 
equal to 20 dBZ at a minimum of 12 nautical miles with a probability of 0.85. Equivalently, the 
probability of not indicating a hazard (missed detection) must be less than 0.15. The second case, as 
defined in section 1.3(b), states that the radar shall not indicate turbulence that corresponds to a standard 
deviation of aircraft g-load excursions of 0.1 g for weather targets with reflectivity greater than or equal to 
20 dBZ at a minimum of 12 nautical miles with a probability of 0.20. Equivalently, the probability of not 
indicating a hazard must be greater than 0.80 for the second case. The basic concepts underlying the 
statistical theory of hypothesis testing are shown in Figure 6. The techniques discussed above are also 
applicable to Class B and C aircraft systems based on performance levels defined in Reference [2]. 
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Figure 6: Calculation of Nuisance and Missed Detection Probabilities Based on Hypothesis Testing 

As indicated in Figure 6, hypothesis HI defines the must not indicate/detect conditions and H2 defines the 
must indicate/detect requirement. A representative example threshold (z t ) for z is also shown in Figure 6. 
It should be noted that regardless of where the threshold is placed, an error of Type 1 (nuisance) or Type 
2 (missed), or both, will occur. The area under the z-PDF (dashed line) to the left of the example 
threshold is the probability of missed detection (i.e. no indication of turbulence) when in fact hypothesis 
H2 is true. The ATDS Working Group’s MOPS requirement is that this estimate should not exceed 0.15. 
Likewise the area under the z-PDF (solid curve) to the right of the example threshold is the probability of 
a nuisance detection (i.e. indication of turbulence) when in fact hypothesis HI is true. In this case, the 
requirement of the MOPS is that this estimate should not exceed 0.20. The z-PDF curves shown in Figure 
6 were computed using Equation (6) developed in Section 4 of this report. The z-PDF’s shown were 
based on the “one-size-fits-all” model and published radar performance data that will be further discussed 
in Sections 6 and 8 of this report respectively. It should be noted, that although the probability density 
functions shown in Figure 6 appear to be Gaussian, in fact they are not. In general, the random variable 
z = xy is not Gaussian, even in the case where the random variables x and y are both Gaussian. 

In order to calculate probabilities defined above, the PDF of the random variable z must first be 
calculated using Equation (6), previously developed in Section 4. Once the z-PDF is known for 0 < z < 
oo, the probability of a nuisance turbulence indication can be calculated using the following equation: 

Pr(z > z t ) = J“p z (z|z > 0,p z = O.lg, a z )dz (33) 

Likewise the probability of a correct turbulence indication can be calculated: 

Pr(z > z t ) = /”p z (z|z > 0 ,p z = 0.3g, a z )dz (34) 

The probability of a missed turbulence indication is the complement of Equation (34). 
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Equation (13) may be used to set the O.lg and 0.3g conditions as defined in the integrands of Equations 
(33) and (34) respectively. If the mean of the aircraft conversion factor x is known as a function of flight 
conditions (see Section 6), then the mean of the radar spectrum width y can be calculated from Equation 
(13) such that the 0. lg and 0.3g design conditions are true. For example, if the mean of x is assumed to be 
the value used in the verification analysis of Section 4, then the mean of y (y is assumed to have been 
pulse volume compensated) required to achieve the 0.3g must indicate condition, assuming the 
normalization factor N = 1, is given by: 


u = °' 3g = 6.435 m/s (35) 

^ y 0.04662 g/m/s V ' 

The variance of y is the key parameter in determining the end-to-end performance characteristics of an E- 
Turb Radar system. The variance of y is a function of spectrum width estimator properties and specific 
spatial and temporal averaging techniques employed by the applicant. 

6. Technical Approach for Hazard Table Generation and “One-Size-Fits- 
All” Model for Class A Aircraft Systems 

As discussed in previous sections, the ATDS Working Group’s MOPS requires conversion of airborne 
radar measurement products to estimated RMS g-loads. Conversion of radar spectrum width to RMS g’s 
is dependent on aircraft gust load response characteristics. Aircraft gust load response characteristics may 
be described by hazard table data, which is a function of aircraft weight and balance, airspeed, and 
altitude. 

The technical approach for the generation of the hazard table for a group of aircraft of similar design is 
presented in Figure 7. The first five boxes of the table are specific to each contributing aircraft of the final 
hazard table product and are labeled as “weight strapped.” The approach taken in this body of work 
addresses the generation of one hazard table for one class of aircraft that can be estimated from a baseline 
set of statistical data. The following describes, in further general detail, each of the contributing parts of 
Figure 7. 
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Figure 7: Technical Approach for Hazard Table Generation and “One-Size-Fits-All” Model for Class A 

Aircraft Systems 


14 





The data collection process begins with the definition of the trim flight conditions of interest for a specific 
aircraft type. The selected conditions focus on the operational flight envelope of the commercial aircraft 
of interest in terms of altitude (typically 5,000 to 40,000 feet), weight (typically operating empty weight 
to maximum gross weight), and center of gravity position (a percentage of mean aerodynamic chord for 
the aircraft type). Gust penetration airspeed or Mach number is selected for each altitude and weight 
condition considered. The selection process of these flight conditions is done in partnership with the 
Original Equipment Manufacturer (OEM), simulator personnel, and project personnel. 

The next key step is the selection of the source for the aircraft response data that will be the primary input 
into the hazard table generation. Previous data collection processes focused on the Level D, FAA 
certified, Full-Flight Simulators available at many larger commercial carrier training facilities or NASA 
research centers. These simulators (Reference [7]) provide the highest level of fidelity in matching the 
real-world aircraft, including its response to a turbulence gust and the modeling of the ship’s auto-pilot 
response to disturbances. Suitable data may also be collected from “desktop” simulators if the same level 
of fidelity is present within the dynamic response model. The benefit of using a desktop simulator versus 
a piloted simulator is the speed at which the data collection process can be accomplished and the cost 
savings associated with this option. 

After a suitable simulator has been selected that meets the needs and requirements of the data collection 
team, data is collected from the simulator by the project team and simulator management personnel that 
will be used in the formation of the hazard table. A data collection matrix is developed from the identified 
trim flight conditions focusing on a range of altitudes, weights, and c.g. positions. The input disturbance 
to the simulator is a vertical gust in the inertial axis of the system with no other ambient atmospheric 
phenomenon present. This isolates the computer’s calculated aircraft response to the gust input, the kernel 
of the aircraft response needed in the hazard table generation. For each point within the data collection 
matrix, a time series response of the aircraft to this gust input is calculated by the aircraft simulator model 
and stored for offline data processing. 

Once the recorded data has been processed and checked for such anomalies as poor trim conditions, 
unsteady flight conditions after the disturbance or noisy signals in general, the data is reduced into a 
subset of the total time response, isolating only the aircraft’s response to the turbulence input. In general, 
significantly more data is collected in each run for diagnostic purposes. A Fast Fourier Transform (FFT) 
is applied at each data point from the data collection matrix for the normal acceleration and the inertial 
vertical gust time series’ response, thereby transforming them into a frequency domain response for the 
specific flight conditions. 

With the individual frequency responses for each resulting time series for each individual data collection 
point in hand, the gust response transfer function is now calculated. This provides the basic information 
required for the eventual generation of a hazard table. 

To calculate the RMS normal load per RMS vertical gust velocity response, a calculation is performed 
that combines the frequency response gust transfer functions with the modeled von Karman turbulence 
spectra. The von Karman spectra and associated scale lengths (20 to 2000 meters) are calculated based on 
the flight conditions for each of the data collection points. The resulting product of RMS normal load per 
RMS vertical gust velocity is in the units of g’s/m/s. A table of values for the range of length scales is 
generated for each aircraft type of interest. A default turbulence length scale of 500 meters is 
recommended for application of the hazard table data. 

The approach outlined above has been successfully applied to eight commercial transport category 
aircraft (Class A) ranging in size from a B737-300 to a B747-400. The format of the hazard table data for 
one selected commercial transport aircraft is shown in Table 2. The dash marks within Table 2 represent 
flight conditions for which a suitable trim solution was not possible for this particular aircraft, altitude, 
weight, and center of gravity position. 
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Table 2: Example Format and Data for a Hazard Table for a Commercial Transport Aircraft (Class A) 


Weight 

(klbs) 

Altitud 

le (kft) 

5 

10 

15 

20 

25 

30 

35 

40 

100 

0.05423 

0.05742 

0.05400 

0.05071 

0.05227 

0.05432 

0.05100 

0.04501 

110 

0.05109 

0.05341 

0.05036 

0.04718 

0.04945 

0.05260 

0.04970 

0.04204 

120 

0.04650 

0.05072 

0.04755 

0.04474 

0.04532 

0.04958 

0.04624 

0.03640 

130 

0.04295 

0.04795 

0.04463 

0.04258 

0.04141 

0.04632 

0.04165 

0.03185 

140 

0.03925 

0.04428 

0.04174 

0.04035 

0.03821 

0.04425 

0.03796 

0.03049 

150 

0.03621 

0.04058 

0.03869 

0.03826 

0.03515 

0.04188 

0.03506 

- 

160 

0.03417 

0.03638 

0.03550 

0.03737 

0.03346 

0.03976 

0.03096 

- 

170 

0.03264 

0.03196 

0.03274 

0.03456 

0.03299 

0.03695 

0.02834 

- 


The additional steps needed for the calculation of “one-size-fits-all” x-PDFs are indicated by the last four 
blocks in Figure 7. The hazard table data for each of the eight aircraft is used to obtain a functional curve 
fit over weight (wing loading) for each aircraft and flight condition (altitude and airspeed). Operational 
weight statistics for a variety of aircraft, provided by participating airlines and OEMs, is then used to 
define weight PDFs for each aircraft considered. Aircraft weight data for over 900,000 takeoffs and 
landings was used to statistically define the PDFs. Using the functional weight curve fit and weight PDFs, 
a Monte-Carlo analysis is conducted and relevant statistics are calculated for the x values (g’s/m/s) for 
each aircraft and flight condition. Finally, the x values are averaged over the eight-aircraft population for 
each flight condition. The final data product for the “one-size-fits-all” design is a PDF of the conversion 
factor for radar second moment observables and is shown in Figure 8 in multiple graphs for the purpose 
of readability. 



x (g's/m/s) 


Note #1 : Units of /j x and a x are g's/m/s 
Note #2: Curves may be modeled as Gaussian 
distributions with indicated // x and cr x . 


Figure 8: Class A Probability Density Functions for “One-Size-Fits-All” Algorithm Applicable for Aircraft 

Wing Foading between 80 and 135 lbs/ft 2 
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Table 3 lists the specific values for each flight condition present within the “one-size-fits-all” design for 
Class A aircraft systems. 

Table 3: Class A Probability Density Functions for “One-Size-Fits-All” Algorithm Applicable for Aircraft 

Wing Loading between 80 and 135 lbs/ft 2 


Flight Condition 

PDF Conversion Factor 

Altitude 

(kft) 

Airspeed 
(kias/Mach No.) 

Mean, ju x 
(g’s/m/s) 

Standard Deviation, cr x 
(g’s/m/s) 

5 

250 

0.04514 

0.004071 

10 

250-290 

0.04662 

0.004256 

15 

250-290 

0.04642 

0.003938 

20 

270-290 

0.04425 

0.003547 

25 

275-290 

0.04378 

0.003866 

30 

275-290 

0.04555 

0.003755 

35 

0.76-0.82 

0.04126 

0.003716 

37-40 

0.76-0.82 

0.03417 

0.003834 


The curves shown in Figure 8 and the data listed in Table 3 may be modeled as Gaussian distributions 
with means and standard deviations as indicated for each flight condition. These probability density 
functions are acceptable for typical jet transport category airplanes with wing loading between 80 and 135 
lbs/fit 2 throughout the entire flight regime. For turbulence systems intended for installation on non-typical 
aircraft with configurations or wing loading outside this range; the impact of configuration and wing 
loading must be considered when calculating the probability of missed detection and false positive 
turbulence indications. 

7. Conversion Factor PDFs for Class B and C Aircraft Systems 

As discussed in previous sections, the Reference [2] requires conversion of airborne radar measurement 
products to estimated RMS g-loads. The objective of this section is to extend and document the x-PDFs 
conversion factors to Class B and C aircraft systems as defined in Reference [2]. The requirement for this 
extension stems from a FAA decision to provide a comprehensive Technical Standard Order (TSO), 
applicable to a broad cross-section of aircraft types, including regional and business jets. 

In order to extend conversion factors to Class B and C systems having wing loading characteristics lower 
than Class A, direct support from aircraft OEMs was sought. Several OEMs were contacted to explore 
their participation in providing required aircraft response data. Two aircraft manufacturers, who produce a 
wide variety of business and regional jet aircraft, agreed to participate in the effort. After discussion with 
the OEMs, nine aircraft models were selected for data acquisition, including six executive business jets 
and three regional transport category jets. The data collection process and subsequent data processing 
techniques employed are consistent with those described by the first five boxes shown in Figure 7 of this 
report. All aircraft selected are modem in-production aircraft or will be in production within 
approximately a year. Some salient technical characteristics of the aircraft selected are summarized in 
Table 4. Values presented in the table below are representative but not exact in all cases. 
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Table 4: Data Collected for Class B and C Aircraft Systems 


Aircraft 

Type 

Wing Area 
(ft 2 ) 

Wing Span 
(ft) 

Maximum Altitude 
(ft) 

Maximum 

Speed 

Wing Foading Range 
(lbs/ft 2 ) 

Biz. Jet 

225 

40 

41,000 

M 0.6 

30-40 

Biz. Jet 

525 

60 

47,000 

M 0.8 

35-55 

Biz. Jet 

525 

60 

51,000 

M 0.9 

45-65 

Biz. Jet 

675 

80 

45,000 

M 0.8 

45-70 

Biz. Jet 

1025 

90 

51,000 

M 0.8 

50-95 

Reg. Jet 

525 

70 

41,000 

M 0.8 

60-100 

Reg. Jet 

725 

80 

41,000 

M 0.8 

60-100 

Reg. Jet 

750 

80 

41,000 

M 0.8 

65-110 

Biz. Jet 

450 

60 

41,000 

M 0.7 

65-105 


Biz. Jet = Executive Business Jet Reg. Jet = Regional Jet 

Based on longitudinal dynamic response data provided by the participating aircraft OEMs, turbulence 
aircraft databases were generated over a representative range of altitudes, airspeeds, and weights for the 
nine aircraft considered. Although the quality of the database is considered good, it is limited as regards 
to development of a “one-size-fits-all” statistical model for Class B and C systems due to the small 
population of aircraft included in each Class. In order to mitigate the impact of the limited population of 
aircraft considered, an alternative approach was developed that enabled the application of the large 
statistical sample of data for Class A aircraft systems presented in Section 6 of this report. The functional 
block diagram shown in Figure 9 illustrates the general technical approach for scaling Class A data to 
Class B and C systems. In order to implement the process outlined in Figure 9 a scaling law is first 
required. Once the scaling law is determined, calculations are made to transform Class A PDF data to 
Class B and C systems specifications. Finally the scaled results for Class B and C systems can be 
compared to those calculated directly from the limited OEM provided data for verification. 



Figure 9: Technical Approach for Generating PDFs for Class B and C Aircraft Systems 

The similarity scaling law derived is based on the well-known fact that the conversion factor x values 
(g’s/m/s) are inversely proportional to wing loading and proportional to airspeed. For example, consider 
two aircraft of vastly different wing loading ranges and assume the conversion factors x (g’s/m/s) for 
aircraft #1 are known and we desire to scale those to aircraft #2. Then we know that 
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(36) 


Xi 


(W/S) 1 


and x? 


(W/S) 2 


where V and W/S represents the airspeed and the wing loading respectively of the vehicle of interest. 
Also, let A 1 and A 2 be factors that renders the above two proportions an equality. Then, 


x x = A x 


(1 W/S)! 


and x 2 = A 2 


(W/s) 2 


(37) 


with the implication that 


*2 


= Xi 


ai rov/shi rwi 

.aJ i(w/s) 2 \ LvJ 


(38) 


The key assumption underlying the similarity scaling technique is A 1 ~ A 2 and is based on the notion that 
conventional fixed-wing commercially-operated aircraft have similar flying and handling qualities. The 
lambda coefficients introduced above are largely dependent on aircraft stability and control parameters, 
which dictates the modal frequency and damping characteristics of aircraft longitudinal dynamic response 
to turbulence. The frequency and damping characteristics must lie in a relatively small range of values in 
order to achieve satisfactory handling qualities for conventional fixed-wing aircraft designs. Further 
details can be found in Reference [8] where the ranges of frequency and damping are expressed in terms 
of iso-contours of constant handling qualities based on pilot opinion. 

Applying the basic ideas presented above, a final similarity scaling law can be written and a verification 
analysis conducted. The functional form of the similarity scaling law for the nine aircraft considered is 
given by: 


x n = X 0 (V 0 (h)) X 


E[(W/S)o\ 
- (W/S)n 


lxp^l,n = 1...9 
J Lv 0 (/i)J 


(39) 


where: 


A Gaussian random variable for Class A systems as defined in Figure 8 

Class A population mean statistic based on over nine hundred thousand flight 
operations 

Speed adjustment ratio based on OEM data collection process for the nine aircraft 
considered 

Wing loading range values for which OEM data was collected for the n th aircraft 

In the above development, subscript zero refers to the data model associated with the population of Class 
A aircraft systems and W /S is defined as wing loading in lbs/ft 2 . The units of x in all cases are g’ s/m/s. 
The index n identifies any one of the nine aircraft defined in Table 4. 

Since x 0 is defined as a Gaussian random variable, then x n is a Gaussian random variable with expected 
value E[x 7 J and standard deviation er(x n ) given by 


*o0o (/0) = 
EKW/S) „] = 

frnCO l 

Lvb(ft) J 

[(W/S) n \ = 


E I A n ] Ex 0 




p?[(W/S)o]l 

X 

TnO) 1 

L (W/S)n 

lv 0 (h)\ 

p?[(VK/S) 0 ]l 

X 

\v n 0)1 

[ (W/S) n 

k(h)J 


(40) 

(41) 
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where fi Xo and o Xq are the mean and standard deviation for Class A systems as defined in Figure 8. Figure 
10 through Figure 14 show comparisons of scaled results E[x\ to those calculated from OEM provided 
response data for two specific aircraft types. Both a Class B and Class C aircraft were selected to 
demonstrate typical results of the verification analysis. 
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Figure 10: Comparison of Scaled and OEM Data for a Class B Aircraft 
at 5,000 ft; 15,000 ft; 25,000 ft; and 35,000 ft 
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Figure 11: Comparison of Scaled and OEM Data for a Class B Aircraft 
at 10,000 ft; 20,000 ft; 30,000 ft; and 40,000 ft 
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Figure 12: Comparison of Scaled and OEM Data for a Class C Aircraft at 10,000 ft 
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Figure 13: Comparison of Scaled and OEM Data for a Class C Aircraft at 20,000 ft 
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Equations (40) and (41) can be used to calculate the class based “one-size-fits-all” x-PDFs for Class B 
and C systems. In order to facilitate these calculations, it is necessary to quantify the class based wing- 
loading terms expressed in the denominator of Equations (40) and (41). These parameters were selected 
as the median value for the class being considered. The final results for the class based x-PDFs are shown 
in Figure 15 and Figure 16 and relevant data are summarized in Table 5 and Table 6. 
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Figure 15: Class B Probability Density Functions for “One-Size-Fits-All” Algorithm Applicable for Aircraft 

Wing Loading between 60 and 100 lbs/ft 2 

Table 5 lists the specific values for each flight condition present within the “one-size-fits-all” design for 
Class B aircraft systems. 

Table 5: Class B Probability Density Functions for “One-Size-Fits-All” Algorithm Applicable for Aircraft 

Wing Loading between 60 and 100 lbs/ft 2 


Flight Condition 

PDF Conversion Factor 

Altitude 

(kft) 

Airspeed 
(kias/Mach No.) 

Mean, //, 
(g’s/m/s) 

Standard Deviation, cr x 
(g’s/m/s) 

5 

250 

0.05473 

0.004936 

10 

250-290 

0.05653 

0.005160 

15 

250-290 

0.05628 

0.004775 

20 

270-290 

0.05365 

0.004301 

25 

275-290 

0.05308 

0.004688 

30 

275-290 

0.05523 

0.004553 

35 

0.76-0.82 

0.05003 

0.004506 

40 

0.76-0.82 

0.04143 

0.004649 
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Figure 16: Class C Probability Density Functions for “One-Size-Fits-All” Algorithm Applicable for Aircraft 

Wing Loading between 30 and 70 lbs/ft 2 

Table 6 lists the specific values for each flight condition present within the “one-size-fits-all” design for 
Class C aircraft systems. 

Table 6: Class C Probability Density Functions for “One-Size-Fits-All” Algorithm Applicable for Aircraft 

Wing Loading between 30 and 70 lbs/ft 2 


Flight Condition 

PDF Conversion Factor 

Altitude 

(kft) 

Airspeed 
(kias/Mach No.) 

Mean, /u x 
(g’s/m/s) 

Standard Deviation, cr, 
(g’s/m/s) 

5 

250 

0.08757 

0.007898 

10 

250-290 

0.09044 

0.008257 

15 

250-290 

0.09005 

0.007640 

20 

270-290 

0.08585 

0.006881 

25 

275-290 

0.08493 

0.007500 

30 

275-290 

0.08837 

0.007285 

35 

0.76-0.82 

0.08004 

0.007209 

40 

0.76-0.82 

0.06629 

0.007438 


The curves shown in Figure 15 and Figure 16 and the data listed in Table 5 and Table 6 may be modeled 
as Gaussian distributions with means and standard deviations as indicated for each flight condition. These 
probability density functions are acceptable for typical airplanes with wing loading between 60 and 1 00 
lbs/fit 2 and 30 and 70 lbs/ft 2 respectively throughout the entire flight regime. For airborne radar turbulence 
systems intended for installation on non-typical aircraft with configurations or wing loading outside this 
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range; the impact of configuration and wing loading must be considered when calculating the probability 
of missed detection and false positive turbulence indications. 

8. Radar Spectrum Width PDFs Based on Published Data 

In order to apply the methodology developed in the previous sections of this report, the radar probability 
density functions (y-random variable) and relevant statistics must be known. An applicant seeking 
certification approval for an E-Turb Radar system should provide the radar measurement statistics and 
probability density functions necessary to demonstrate compliance for the specific radar performance 
characteristics and configuration considered. Radar manufacturers typically employ simulation techniques 
for determining radar signal return time series from turbulent weather and radar signal processing for 
Doppler second moment estimation. However, manufacturers consider details regarding algorithms, range 
and cross range spatial filtering techniques, specific radar parameters, and resulting performance data as 
proprietary. Therefore, in order to perform calculations to demonstrate the methodology developed in the 
previous sections, with performance results, a public domain source of required data and spectrum width 
statistics was sought. 

An available public data source for radar spectrum width statistics for modeling the y-probability density 
function can be found on page 138 (Fig. 6.6) of Doviak & Zmic (Reference [9]). The FAA-ATDS 
working group has agreed that these data are representative of airborne radar Doppler second moment 
estimator performance, and is suitable for demonstrating the methodology developed in this report. The 
data found in Reference [9] are based on the fundamental assumption that the distribution of Doppler 
frequency of turbulent weather weighted by the antenna beam illumination and weather reflectivity is 
described by a Gaussian function. The performance data shown in Fig 6.6 of Reference [9] is based on 
autocovariance processing (pulse-pair) estimator of spectrum width for a single power weighted spectrum 
width measurement of narrow width relative to Nyquist frequency. The statistical data shown in Fig 6.6 of 
Reference [9] is in normalized form and can be rendered dimensional by choosing specific radar system 
parameters. The parameters selected, which are typical for airborne radars, are defined below: 

c = speed of light, 3x10 s m/s 

/ = frequency, 9.3x1 0 9 Hz 

prf = pulse repetition frequency (pulses/second), 3000 Hz 

A = c/ f = wavelength, 0.0323 m 

T s = 1 /prf = time between uniform samples, 3.333xl0' 4 s 

v a = 1/ (4 Tf) = unambiguous velocity, 24.23 m/s 

S/N = single pulse signal- to-noise ratio, ~ 15 dB 

Dimensional data for the standard deviation of estimated spectrum width, plotted as a function of mean 
spectrum width estimate, is shown in Figure 1 7. 


25 



4 


Ky) 

(m/s) 


3 


2 


1 


0 1 ‘ 1 ‘ 1 ‘ 1 ‘ 1 ‘ 1 ‘ 1 ‘ 1 ‘ 1 ‘ 1 

123456789 10 

Mean Spectral Width Estimate , y (m/s) 



Figure 17: Standard Deviation of Autoeovariance (Pulse Pair) Estimator vs. Mean Spectral Width 

The data shown in Figure 17 are based on the radar parameters and assumptions stipulated above and M 
refers to the number of pulses processed by the estimator. The y-probability density function can now be 
written as: 

/,()'|y>0)= J5 ^ e xp[-i(fg) 2 ] (42) 

where: 

u(y) = Standard Deviation of y 

y = Mean of y 

1 

C = Normalizing Constant, — 

1-Fy(0) 

9. Application of Defined Methodology with Results for Class A Aircraft 
Systems 

The following text demonstrates the developed end-to-end methodology for calculating the must indicate 
and the must not indicate probabilities as defined in the previous sections of this report. Application of the 
“one-size-fits-all” model as well as a comparison to a “weight-strapped” implementation for the A320- 
200 and B777-200 aircraft is considered for a variety of flight conditions. Statistical performance results 
for representative E-Turb Radar system parameters are calculated, and results compared for the two 
systems implementations. Statistical performance results are presented in the form of a curve called the 
receiver operating characteristic. The terms of reference for the analysis are as follows: 

• Two diverse aircraft types (A320-200 & B777-200) are selected. 

• Extreme operating weights (light & heavy) are used for each aircraft type. 
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• For each aircraft type and weight, two different flight conditions (altitude and airspeed) are 
considered (low and slow, high and fast). 

• Conversion factor x-values for weight-strapped implementation are based on hazard table data 
previously developed. 

• For the “one-size-fits-all” implementation, the x-PDFs are defined in Figure 8 of Section 6. 

• The same radar characteristics/parameters are used throughout the analysis. The specific radar 
parameters and y-PDFs are defined in Section 8. A 15 dB signal to noise ratio (S/N) is assumed, 
and M = 8 pulses selected for spectrum width estimator (pulse-pair) processing. 

All of the calculations of probabilities are performed at an assumed 20 nautical mile radar range. At this 
assumed range, the pulse volume compensation factor (k) approaches a value of approximately 1 over a 
broad range of radar pulse lengths and antenna sizes (see Appendix I). Given the above terms of 
reference, specific numerical data needed for the calculation is shown in Table 7, where x° represents the 
hazard table value for a specific aircraft, weight, and airspeed configuration. 


Table 7: Weight Strapped x° Values 


A320-200 

B777-200 

Weight (klbs) 

Altitude 

Airspeed 

x° (g’s/m/s) 

Weight (klbs) 

Altitude 

Airspeed 

x° (g’s/m/s) 

90 

h = 5 kft 
V = 250 kias 

0.0611 

360 

h = 5 kft 
V = 250 kias 

0.0564 

170* 

h = 5 kft 
V = 250 kias 

0.0386 

640* 

h = 5 kft 
V = 250 kias 

0.0358 

90 

h = 35 kft 
M = 0.76 

0.0564 

360 

h = 37 kft 
M = 0.82 

0.0521 

160* 

h = 35 kft 
M = 0.76 

0.0324 

480* 

h = 37 kft 
M = 0.82 

0.0442 


Data required to support the calculations for the “one-size-fits-all” model are provided in Figure 8 of 
Section 6 and Figure 17 of Section 8. These data, for the specific flight conditions selected, are shown in 
Table 8 below, including the radar mean spectrum width required to set the must indicate and must not 
indicate necessary conditions. The numerical data shown in Table 8 allow for complete specification of 
the x PDFs as defined in Figure 8 of Section 6 and y PDFs as defined by Equation (42). For the 
conditions stipulated above, these x and y PDFs enable direct calculation of the hazard metric z-PDF 
defined by Equation (6). 


Table 8: Data Required for Calculating Must Indicate and Must Not Indicate Probabilities for the “One-Size- 

Fits-AU” Model 


Condition 

Altitude 

Airspeed 

kx (g’s/m/s) 

a x (g’s/m/s) 

y (m/s) 

Oy (m/s) 

Must Not 
Indicate 
kz = O.lg 

h = 5 kft 
V = 250 kias 

0.04514 

0.004071 

2.215 

1.159 

h = 35 kft 
M = 0.78 -0.82 

0.04126 

0.003716 

2.424 

1.209 

Must Indicate 
kz = 0.3g 

h = 5 kft 
V = 250 kias 

0.04514 

0.004071 

6.646 

2.403 

h = 35 kft 
M = 0.78 -0.82 

0.04126 

0.003716 

7.271 

2.609 


Value represents the maximum weight for this aircraft type at this speed and altitude. 
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The probability of correct detection (must indicate) of turbulence conditions plotted as a function of a 
probability of nuisance detection (must not indicate) is shown in Figure 18 through Figure 21. The “one- 
size-fits-all” performance results shown in Figure 18 through Figure 21 were calculated using Equations 
(33) and (34) as was the weight strapped results for a range of positive z-threshold values. The results 
shown in the figures permit a direct comparison of “one-size-fits-all” and weight strapped system 
performance for most extreme operating weights at the indicated flight conditions. 


Receiver Operating Characteristic 



Conditions: 
h = 5 kft 
V = 250 kias 
c.g. =40% mean chord 
Radar Range = 20 NM 
S/N = 15 dB 

Note: 

170 klbs is the maximum 
weightatthis speed and 
altitude. 

Point Threshold . z t 

1 0.120 g 

2 0.150 g 

3 0.175 g 

4 0.200 g 

5 0.230 g 


Figure 18: Comparison of “One-Size-Fits- All” Performance and A320-200 with Weight Strapped Results for 
Extreme Operating Weights at Indicated Flight Conditions (5,000 ft) 
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Receiver Operating Characteristic 



Conditions: 

h = 5 kft 

V = 250 kias 

c.g. = 35% mean chord 

Radar Range = 20 NM 

S/N = 15 dB 

Note: 

640 klbs is the maximum 
weightatthis speed and 
altitude. 

Point Threshold . z t 

1 0.120 g 

2 0.150 g 

3 0.175 g 

4 0.200 g 

5 0.230 g 


Probability of Nuisance Detection 


Figure 19: Comparison of “One-Size-Fits- All” Performance and B777-200 with Weight Strapped Results for 
Extreme Operating Weights at Indicated Flight Conditions (5,000 ft) 


Receiver Operating Characteristic 



Conditions: 

h = 35 kft 

V = Mach 0.76 

c.g. = 40% mean chord 

Radar Range = 20 NM 

S/N = 15 dB 

Note: 

160 klbs is the maximum 
weightatthis speed and 
altitude. 

Point Threshold . z t 

1 0.120 g 

2 0.150 g 

3 0.175 g 

4 0.200 g 

5 0.230 g 


Probability of Nuisance Detection 


Figure 20: Comparison of “One-Size-Fits- All” Performance and A320-200 with Weight Strapped Results for 
Extreme Operating Weights at Indicated Flight Conditions (35,000 ft) 
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Receiver Operating Characteristic 



Conditions: 

h = 35 kft 

V = Mach 0.82 

c.g. = 35% mean chord 

Radar Range = 20 NM 

S/N = 15 dB 

Note: 

480 klbs is the maximum 
weightatthis speed and 
altitude. 

Point Threshold . z t 

1 0.120 g 

2 0.150 g 

3 0.175 g 

4 0.200 g 

5 0.230 g 


Probability of Nuisance Detection 


Figure 21: Comparison of “One-Size-Fits- All” Performance and B777-200 with Weight Strapped Results for 
Extreme Operating Weights at Indicated Flight Conditions (35,000 ft) 

Note that the “one-size-fits-all” curves shown in the above figures are identical for the same specific 
flight condition considered and independent of aircraft type at that flight condition. This observation is 
consistent with the “one-size-fits-all” design philosophy, as long as the aircraft is in the class of large 
commercial transports with wing loading in the range of 80 - 135 lbs/ft 2 (Class A). The results shown in 
Figure 18 through Figure 21 clearly demonstrate robustness of the “one-size-fits-all” algorithm statistical 
design philosophy when compared to the more correct “weight strapped” implementation. For the cases 
considered in the analysis, there is no violation of the 85% correct turbulence indication versus 20% 
nuisance indication performance criteria, provided the proper system threshold is selected. 

Also shown in Figure 18 through Figure 21 are results for five point set threshold values ranging from 
0. 12g to 0.23g. The point indicated by the diamond symbol in the above figures represents an “optimal” 
threshold, assuming the system cost of a missed correct turbulence indication is 2.5 times that of a 
nuisance indication. The optimal threshold calculation is based on the Bayes Criterion the details of which 
can be found in Reference [10], and assumes a prior probability of 0.5 that hypothesis FI2 as defined in 
Figure 6 is true. The numerical value optimal threshold was found to be approximately 0.157 g for both 
flight conditions considered in the analysis, thus indicating low sensitivity for the “one-size-fits-all” 
implementation. 

10. Summary and Conclusions 

The objective of the research developed and presented in this document was to statistically assess 
turbulence hazard detection performance employing airborne pulse Doppler radar systems. The FAA 
certification methodology for forward-looking airborne turbulence radars will require estimating the 
probabilities of missed and false hazard indications under operational conditions. Analytical approaches 
must be used due to the near impossibility of obtaining sufficient statistics experimentally. This report 
describes an end-to-end analytical technique for estimating these probabilities for E-Turb Radar systems 
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under noise-limited conditions, for a variety of aircraft types, as defined in Reference [2], This technique 
provides for one means, but not the only means, by which an applicant can demonstrate compliance to the 
FAA directed ATDS Working Group performance requirements. Turbulence hazard algorithms were 
developed that derived predictive estimates of aircraft hazards from basic radar observables. These 
algorithms were designed to prevent false turbulence indications while accurately predicting areas of 
elevated turbulence risks to aircraft, passengers, and crew; and were successfully flight tested on a NASA 
B757-200 and a Delta Air Lines’ B737-800. Application of this methodology for calculating the 
probability of missed and false hazard indications, taking into account the effect of the various algorithms 
used, is demonstrated for representative transport aircraft and radar performance characteristics. 
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Appendix I 

Calculation of Pulse Volume Compensation Factor 


The following defines the symbology used within this appendix: 
c = Speed of light, 3xl0 8 m/s 
E (fc) = Turbulence energy spectra as a function of k 
k = Turbulence wave number (rad/m) 

L = Turbulence scale length (m) 

R = o r /o d , characteristic range (m) 

r = Radar range (m) 

jit = a! L/o r (non-dimensional), where a' = 1.339 

6 1 = One-way antenna beam width between half-power points (rad) 

a = RMS turbulence intensity of the one-dimensional von Karman energy spectra (m/s) 

Oq = Transverse 2 nd central moment of a two-way Gaussian beam illumination function 1 
(rad) 

o r = Radial 2 nd central moment of a two-way Gaussian beam illumination function 2 (m) 

<J V = Doppler velocity spectrum width (m/s) 
r = Radar pulse width (s) 

The radar pulse volume filtering phenomenon was introduced and discussed in Section 3 of the main body 
of the report. Specific techniques for compensating these effects are presented in this appendix. 
Specifically, a theoretical relationship between radar spectrum width (2 nd moment measurements) and 
turbulent velocity point variance must be defined. In order to develop this relationship, the following 
assumptions are made: 

1. The turbulence wind field is characterized by the universe of realizations based on von Karman 
energy spectra. The turbulent wind field is assumed homogeneous and isotropic. 

2. It is assumed that the reflectivity within the radar resolution volume is uniform, with adequate 
signal-to-noise for reliable Doppler processing. 

3. A Gaussian two-way beam illumination function is assumed with one parameter transverse to and 
the other along the radar beam. 

4. The radar observable is assumed to be a spectrum width product with all spectrum width 
broadening artifacts removed. 

5. A 9.3 GHz X-band radar is assumed with matching receiver bandwidth for a transmitted 
rectangular pulse. 

6. A two-way circular symmetric antenna pattern is assumed with no antenna losses and a diameter 
of 28 inches. 


1 Equation 5.67, p89. Doviak, Richard J. and Zrinc, Dusan S. Doppler Radar and Weather Observations , Academic 
Press, San Diego, California, 1984. 

2 Equation 5.68, p89. Doviak, Richard J. and Zrinc, Dusan S. Doppler Radar and Weather Observations , Academic 
Press, San Diego, California, 1984. Assumes a rectangular transmitted pulse and a matched Gaussian receiver 
frequency response. 
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For the conditions stipulated above it has been shown 1 , with additional prior elaboration by others 4 , that 
the connection between radar Doppler spectrum width and turbulence energy spectra is given by an 
indefinite integral: 

(o'v 00) = - jj^f) [l - exp (- a 2 (k 2 + k 2 ) - b 2 k 2 )\dk x dk y dk z (I-l) 

The above equation expresses the ensemble average of the radar spectrum width at a range r, in terms of 
turbulent energy spectra E(k ) and radar system parameters. The subscripted k's indicate the components 
of the turbulent spatial wave numbers in units of rad/m and k (not to be confused with the k defined in 
Section 3) is the magnitude of the wave number vector. For a specific radar and antenna design the 
parameters a and b are constants and are defined as: 


vQ 

a — ra g — (8ln4)0 . s 

(1-2) 

b = a r = 0.35 (f) 

(1-3) 


If a specific form of turbulent energy spectrum is known, and a particular radar configuration assumed, 
then estimates of radar spectrum width can be made by calculating Equation (I-l). In order to facilitate 
calculation of Equation (I-l), it is convenient to introduce polar spherical coordinates in k - wave number 
space, where the coordinate system is rotated such that the z-axis is along the radar radial line of sight. 
For this coordinate system alignment, direct integration over the polar spherical angles can be carried out 
resulting in the following equations. 

(o-2(r)> = lf™E(k) [l - e' a2k2 M(^,(a 2 - b 2 )/c 2 )] dk for a > b (1-4) 

(er 2 (r)> = ^ /“ E(k) [l - e ~ b2R2 M (2,^, (b 2 - a 2 )/c 2 )] dk for b > a (1-5) 


Where k 0 = — and /L 0 is equal to the turbulence “outer scale” wavelength. 

Aq 

In the above equations, M(a, y, () is the confluent hypergeometric function which is formally defined as: 


M(a,y, 0 


71= co 

r(y) r(n + a) f; n 

T(a) i—k T(n + y) n! 

71 = 0 


( 1 - 6 ) 


The above sum is a convergent series for all positive values of the three arguments. 

A specific form of turbulent energy spectra, E(k), is needed in order to evaluate Equations (1-4) and (1-5). 
It is assumed that E ( k ) is given by the von Karman spectra, which are functionally defined as: 


E{k) = 


55 o 2 L 


( a'LkY 


9 n [1 + (a'Lk) 2 ] 17 / 6 
Where a is the RMS turbulence intensity (m/s) and L is the turbulence scale length (m). 


(1-7) 


3 Chapter 10. Doviak, Richard J. and Zrinc, Dusan S. Doppler Radar and Weather Observations , Academic Press, 
San Diego, California, 1984. 

4 Frisch, A. S. and Clifford, S. F. “A Study of Convection Capped by a Stable Layer Using Doppler Radar and 
Acoustic Echo Sounders,” Journal of Atmospheric Sciences, Volume 31, No. 6, 1974. 
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Substitution of Equation (1-7) into Equations (1-4) and (1-5) and assuming k 0 = 0 yields: 



( 1 - 8 ) 


(1-9) 


for R <r 

Where the independent variable of integration x (not to be confused with x defined in Section 3 of the 
main body of this report) is defined as x = a? k 2 . 

All other parameters indicated in Equations (1-8) and (1-9) are defined in the nomenclature section of this 
appendix. The authors have not found Equations (1-8) and (1-9) in the open literature, although they were 
developed using methods outlined in Footnote #1 of Appendix I, and the authors believe these equations 
offer distinct numerical computational advantages. The specific form of the energy spectra employed in 
the above development is known to contain the Kolmogorov inertial subrange, in which energy cascades 
from large eddies to smaller eddies, with eventual dissipation by viscous forces. However, the above 
derivation is not limited to this assumption, which is typical practice within the industry, since larger 
wavelengths can have significant impact on turbulence response of aircraft depending on speed and other 
design factors. 

Figure 1-1 shows the pulse volume compensation factor calculation results using Equations (1-8) and (I- 
9) plotted as a function of radar range for several selected radar pulse lengths. The specific parameters 
selected for the calculation are: -3 dB antenna beam width of 3.8 deg, turbulence length scale of 500 m, 
turbulence outer scale wave number of zero, and four radar pulse lengths were selected. 
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Figure I - 1: Pulse Volume Compensation Factor versus Radar Range 
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Note that the pulse volume compensation factor k as defined in Section 3 of the main body of this report 
is given by: 


1 

k(r, t, ...) = 

vWoo) ( I ' 1 °) 

a 

For radar ranges greater than 20-25 nautical miles k takes on values of approximately one. However, in 
the radar near field of view, where the resolution volume of the radar is of smaller spatial dimension, 
k > 1. 
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Appendix II 

Inclusion of Radar Pulse Volume Compensation Factor in the Governing Probability Equations 


If an applicant seeking certification approval elects to include radar pulse volume compensation for their 
system implementation, then the governing probability equations presented in the body of this report must 
be modified. The necessary modifications can be accomplished by employing probability density function 
transformation techniques found in Reference [6]. For example: 

If is a random variable and! = a( + b then fx(A) = ~f} provided the equation 
A = a( + b has a single solution of<f = for every A where a and b are constants. 

Consider the “one-size-fits-all” model first. If the uncompensated process z = xy has a z-PDF given by 
Equation (6): 


f I f °o/y(y)/x(z/y) / TT 

p z (z|z > o ) = N f 0 — dy (II-l) 

then using the above transformation, the PDF for the compensated process z = kxy is given by: 

The compensation factor k is a constant at a specified radar range and choice of radar system parameters. 
Using Equation (II-2) the mean and variance of z can be directly calculated and are given by the 
following: 


kz = J 0 °° z Vz (z)dz = Nkfi x fi y (H-3) 

a z = J 0 °°z 2 p z (z)dz -nl = ( Nk Y [(cr x cr y ) 2 + (p x cr y ) 2 + (p y cr x ) 2 ] (D-4) 

For the pulse volume compensated “weight strapped” case, the system process is defined as z = kx°y 
where x° is the hazard table value for a specific airplane, weight, and flight condition. The 
uncompensated (7c = 1) z-PDF as defined by Equation (29) is: 

p z (z|z > 0) = ±f y (J) where N = (11-5) 

Using the transformation technique outlined above, the compensated z-PDF is given by: 

P Z (Z|Z > °) = gy (II-6) 

where the mean and variance are given by : 

Pz = J” zp z (z)dz = Nkx () fly (11-7) 

°z = J 0 ™ z 2 p z (z)dz — p| = (Nkx°a y ) 2 (11-8) 
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